Content-Learning Correlations in Spoken Tutoring Dialogs at Word, Turn, and Discourse Levels

We study correlations between dialog content and learning in a corpus of human-computer tutoring dialogs. Using an online encyclopedia, we first extract domainspecific concepts discussed in our dialogs. We then extend previously studied shallow dialog metrics by incorporating content at three levels of granularity (word, turn and discourse) and also by distinguishing between students’ spoken and written contributions. In all experiments, our content metrics show strong correlations with learning, and outperform the corresponding shallow baselines. Our word-level results show that although verbosity in student writings is highly associated with learning, verbosity in their spoken turns is not. On the other hand, we notice that content along with conciseness in spoken dialogs is strongly correlated with learning. At the turn-level, we find that effective tutoring dialogs have more content-rich turns, but not necessarily more or longer turns. Our discourse-level analysis computes the distribution of content across larger dialog units and shows high correlations when student contributions are rich but unevenly distributed across dialog segments.